Papers with Cross-Lingual Transfer
Investigating the Potential of Task Arithmetic for Cross-Lingual Transfer (2024.eacl-short)
Copied to clipboard
| Challenge: | Massively multilingual Transformer-based models (MMTs) can learn representations which have a degree of cross-lingual alignment despite being trained using purely unsupervised objectives. |
| Approach: | They propose a modular approach to cross-lingual transfer using task arithmetic . they show that modularity can be achieved even with full model fine-tuning . |
| Outcome: | The proposed approach shows strong performance on multilingual benchmarks encompassing both high-resource and low-resourced languages. |
The Effects of Surprisal across Languages: Results from Native and Non-native Reading (2022.findings-aacl)
Copied to clipboard
| Challenge: | Context-dependent predictive processes have been proposed as a core component of the human cognitive system. |
| Approach: | They extract surprisal estimates from mBERT and assess their predictive power on the MECO corpus, a cross-linguistic dataset of eye movement behavior in reading. |
| Outcome: | The proposed model is based on a cross-linguistic dataset of eye movement behavior in reading. |
Cross-Lingual Transfer of Cultural Knowledge: An Asymmetric Phenomenon (2025.acl-short)
Copied to clipboard
| Challenge: | Existing studies evaluate whether large language models handle global cultural diversity . however, mechanisms behind cultural knowledge acquisition remain unexplored . |
| Approach: | They propose an interpretable framework to study cultural knowledge transfer in large language models . they observe bidirectional cultural transfer between English and other high-resource languages . |
| Outcome: | The proposed framework ensures training data transparency and controls transfer effects. |
The SUMMA Platform: A Scalable Infrastructure for Multi-lingual Multi-media Monitoring (P18-4)
Copied to clipboard
| Challenge: | The SUMMA Platform is a highly scalable distributed architecture for monitoring a large number of media broadcasts in parallel, with a lag behind actual broadcast time of at most a few minutes. |
| Approach: | The open-source SUMMA Platform is a highly scalable distributed architecture for monitoring a large number of media broadcasts in parallel . it offers a fully automated media ingestion pipeline capable of recording live broadcasts, detection and transcription of spoken content, translation of all text (original or transcribed) into English, recognition and linking of Named Entities, topic detection, clustering and cross-lingual multi-document summarization of related media items and extraction and storage of factual claims in these news items. |
| Outcome: | The SUMMA Platform is a highly scalable distributed architecture for monitoring a large number of media broadcasts in parallel, with a lag behind actual broadcast time of at most a few minutes. |
Automating Interlingual Homograph Recognition with Parallel Sentences (2022.findings-aacl)
Copied to clipboard
| Challenge: | Existing methods for interlingual homograph recognition require linguistic knowledge and massive annotation work. |
| Approach: | They propose an automatic interlingual homograph recognition method based on cross-lingual word embedding similarity and co-occurrence of form-identical words in parallel sentences. |
| Outcome: | The proposed method can make accurate predictions across languages. |
Beyond the English Web: Zero-Shot Cross-Lingual and Lightweight Monolingual Classification of Registers (2021.eacl-srw)
Copied to clipboard
Liina Repo, Valtteri Skantsi, Samuel Rönnqvist, Saara Hellström, Miika Oinonen, Anna Salmela, Douglas Biber, Jesse Egbert, Sampo Pyysalo, Veronika Laippala
| Challenge: | Existing studies on register classification for web documents have limited results due to skewed datasets and low performance. |
| Approach: | They propose two new register-annotated corpora for French and Swedish . they show that deep pre-trained language models perform strongly in these languages . |
| Outcome: | The proposed models outperform existing models in English and Finnish and can match or surpass existing models. |
Robust Cross-Lingual Hypernymy Detection Using Dependency Context (N18-1)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual hypernymy detection are sparse and can be trained on related languages with negligible loss of performance. |
| Approach: | They propose a family of unsupervised approaches for cross-lingual hypernymy detection which learns sparse, bilingual word embeddings based on dependency contexts. |
| Outcome: | The proposed approach significantly improves performance on this task, compared to approaches based only on lexical context. |
S4-Tuning: A Simple Cross-lingual Sub-network Tuning Method (2022.acl-short)
Copied to clipboard
| Challenge: | Existing multilingual pre-trained language models allow to adapt to target languages with only few labeled examples. |
| Approach: | They propose a simple cross-lingual sub-network tuning method that detects the most essential sub-netzwork for each target language and updates it during fine-tuning. |
| Outcome: | The proposed method improves on three multi-lingual tasks involving 37 different languages. |
Disentangling Linguistic Relatedness from Task Alignment in Cross-Lingual Transfer (2026.acl-srw)
Copied to clipboard
| Challenge: | Large language models (LLMs) have advanced natural language processing, yet their benefits remain concentrated in English and a small number of high-resource languages. |
| Approach: | They fine-tuned large language models (4B–671B parameters) on Arabic and evaluated zero-shot reading comprehension on Semitic languages and non-Semitic controls. |
| Outcome: | The results show that models with weak baselines improve across all languages, whereas strong-baseline models show only marginal gains regardless of language family. |
Cross-Lingual Knowledge Transfer for Clinical Phenotyping (2022.lrec-1)
Copied to clipboard
Jens-Michalis Papaioannou, Paul Grundmann, Betty van Aken, Athanasios Samaras, Ilias Kyparissidis, George Giannakoulas, Felix Gers, Alexander Loeser
| Challenge: | Current models for clinical phenotyping are limited to clinical notes written in English due to the large amount of labeled and unlabeled clinical text resources. |
| Approach: | They propose to use translation-based methods with domain-specific encoders and cross-lingual encoder plus adapters to perform this task for clinics that do not use the English language. |
| Outcome: | The proposed strategies outperform the state-of-the-art models for clinics that do not use the English language and have a small amount of in-domain data available. |
Cross-lingual Transfer of Monolingual Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on cross-lingual learning using multilingual models cast doubt on shared vocabulary and joint pre-training . et al. (2005) show that model knowledge learned in the source language enhances the learning of the target language independently of language proximity. |
| Approach: | They propose a method for transferring monolingual models to other languages through continuous pre-training and investigate their results in English. |
| Outcome: | The proposed method outperforms a model trained from scratch in the GLUE benchmark for English . it shows that model knowledge from the source language enhances the learning of syntactic and semantic knowledge in english. |
Analyzing the Intensity of Complaints on Social Media (2022.findings-naacl)
Copied to clipboard
| Challenge: | Prior studies on identifying the existence or the type of complaints focus on building automatic classification models for identifying complaints. |
| Approach: | They propose to measure the intensity of complaints from text using Best-Worst Scaling method to estimate the popularity of posts on social media. |
| Outcome: | The proposed model can estimate the popularity of complaints on social media with best-worst scaling (BWS) method. |
Cross-lingual Transfer Learning with Data Selection for Large-Scale Spoken Language Understanding (D19-1)
Copied to clipboard
| Challenge: | Existing approaches to improve cross-lingual transfer learning on spoken language are pre-train on all available supervised data from another language. |
| Approach: | They propose a language model based source-language data selection method for cross-lingual transfer learning in spoken language understanding. |
| Outcome: | The proposed method reduces training time and improves model performance on spoken language understanding. |
Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to adapt pretrained Large Language Models to new lower-resource languages are limited to English. |
| Approach: | They propose to combine cross-lingual instruction-tuning with additional monolingual pretraining to adapt LLMs to new lower-resource languages. |
| Outcome: | The proposed model is the first open-source instruction-following LLM for Estonian . the proposed model improves commonsense reasoning and multi-turn conversation capabilities . |
Automatically Creating a Lexicon of Verbal Polarity Shifters: Mono- and Cross-lingual Methods for German (C18-1)
Copied to clipboard
| Challenge: | a large number of verbal polarity shifters are available for multiple languages, but only English has a sizable lexicon of them. |
| Approach: | They use methods to create large lexicon of verbal polarity shifters in germany . they bootstrap annotated verbs with a supervised classifier and apply them to German . |
| Outcome: | The proposed method is able to create a large lexicon of verbal polarity shifters in germany . it reduces annotation effort by leveraging cross-lingual information from the English lexico . |
Cross-lingual Continual Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing multi-lingual representations such as the one-hop transfer learning pipeline are difficult to adapt to new languages. |
| Approach: | They propose a cross-lingual continuum learning paradigm that evaluates continuous learning approaches that adapt to emerging data from different languages. |
| Outcome: | The proposed model can be used to adapt to new languages in a sequential manner. |
On Difficulties of Cross-Lingual Transfer with Order Differences: A Case Study on Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing studies on crosslingual transfer have focused on word-level information sharing, but words are not independent in sentences; their combinations form larger linguistic units, known as context. |
| Approach: | They propose to use orderagnostic models to transfer word order to distant languages . they train dependency parsers on an English corpus and evaluate their transfer performance on 30 other languages. |
| Outcome: | The proposed model performs better on languages with different word orders than on other languages. |
Cross-Lingual Phrase Retrieval (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual phrase retrieval learn word or sentence representations in word or sentences. |
| Approach: | They propose a cross-lingual phrase retrieval model that extracts phrase representations from unlabeled example sentences. |
| Outcome: | The proposed model outperforms state-of-the-art methods on a large-scale cross-lingual phrase retrieval dataset, showing it can perform in an unseen language pair during training. |
Language Anisotropic Cross-Lingual Model Editing (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing work studies monolingual model editing, which lacks cross-lingual transferability to perform editing simultaneously across languages. |
| Approach: | They propose a framework to naturally adapt monolingual model editing approaches to the cross-lingual scenario using parallel corpus. |
| Outcome: | The proposed framework adapts monolingual model editing approaches to the cross-lingual scenario using parallel corpus and amplifies different subsets of parameters for each language. |
Towards Instance-Level Parser Selection for Cross-Lingual Transfer of Dependency Parsers (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods of cross-lingual parser transfer focus on predicting the best parsers for a low-resource target language globally. |
| Approach: | They propose a cross-lingual parser transfer paradigm that uses instance-level parsers to predict the best parsing for a target language at treebank level. |
| Outcome: | The proposed model outperforms existing models on 13/20 and 14/20 test languages. |
Infant Word Comprehension-to-Production Index Applied to Investigation of Noun Learning Predominance Using Cross-lingual CDI database (L18-1)
Copied to clipboard
| Challenge: | Existing theories suggest nouns should predominate verbs in children's word learning . |
| Approach: | They define a measure called the comprehension-to-production index to investigate whether nouns have predominance over verbs in children's word learning. |
| Outcome: | The proposed measure indicates noun predominance in word learning by children . it could provide clues for engineering solutions for teaching words to computers . |
Cross-lingual Named Entity Corpus for Slavic Languages (2024.lrec-main)
Copied to clipboard
| Challenge: | This work presents a corpus manually annotated with named entities for six Slavic languages . |
| Approach: | They propose to manually annotate a corpus of names for six Slavic languages . they use a transformer-based neural network architecture to train multilingual models . |
| Outcome: | The corpus consists of 5,017 documents on seven topics . each entity is described by a category, a lemma, and a unique cross-lingual identifier. |
PLUG: Leveraging Pivot Language in Cross-Lingual Instruction Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Instruction tuning has advanced large language models (LLMs) but its application in lower-resource languages faces challenges due to the imbalanced foundational abilities of LLMs across different languages. |
| Approach: | They propose a pivot language guided generation approach that utilizes a high-resource language as the pivot to enhance instruction tuning in lower-resourced languages. |
| Outcome: | The proposed approach improves instruction-following abilities of LLMs by 29% on average compared to directly responding in the target language alone. |
Evaluating the Robustness and Accuracy of Text Watermarking Under Real-World Cross-Lingual Manipulations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a study focuses on evaluating watermarking methods for the English language . the literature for evaluating cross-lingual watermarks is scarce . |
| Approach: | They evaluate representative watermarking methods in four different languages . they examine the quality of text under different watermark procedures . |
| Outcome: | The proposed method is compared with other evaluation methods in four different languages. |
Learning How to Active Learn by Dreaming (P19-1)
Copied to clipboard
| Challenge: | Recent active learning methods are limited when the data distribution of learning problems vary. |
| Approach: | They propose a wake-and-dream-based active learning method that learns the AL policy directly on the target domain of interest by using wake and dream cycles. |
| Outcome: | The proposed method improves on cross-domain and cross-lingual tasks. |
Measuring Cross-lingual Transfer in Bytes (2024.naacl-long)
Copied to clipboard
| Challenge: | Multilingual pretraining models can transfer knowledge to target languages with minimal or no examples . underlying mechanisms for this transfer remain unclear, with hypotheses ranging from language contamination to syntactic similarity. |
| Approach: | They conducted an experiment to investigate whether multilingual models transfer knowledge to target languages . they found that models initialized from diverse languages perform similarly to a target language . |
| Outcome: | a new study shows that models initialized from diverse languages perform similarly to a target language in a cross-lingual setting. |
Medical Crossing: a Cross-lingual Evaluation of Clinical Entity Linking (2022.lrec-1)
Copied to clipboard
Anton Alekseev, Zulfat Miftahutdinov, Elena Tutubalina, Artem Shelmanov, Vladimir Ivanov, Vladimir Kokh, Alexander Nesterov, Manvel Avetisian, Andrei Chertok, Sergey Nikolenko
| Challenge: | Existing approaches to medical entity linking are limited in terms of data volume and languages. |
| Approach: | They propose to use clinical reports, clinical guidelines, and medical research papers to evaluate cross-lingual medical entity linking. |
| Outcome: | The proposed model outperforms existing models on clinical reports, clinical guidelines, and medical research papers. |
Contrastive Language Adaptation for Cross-Lingual Stance Detection (D19-1)
Copied to clipboard
| Challenge: | Current approaches to fact-checking are time-consuming and tedious. |
| Approach: | They propose a novel approach which leverages labeled data in one language to identify relative perspective of a document with respect to a claim in a different target language. |
| Outcome: | The proposed approach can deal with the challenge of limited labeled data in the target language. |
Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Matter (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior work on cross-lingual transfer often focuses on a small set of languages from a few language families and/or a single task. |
| Approach: | They analyze cross-lingual transfer for 263 languages from a wide variety of language families . they include three popular NLP tasks: POS tagging, dependency parsing, topic classification . |
| Outcome: | The proposed approach is based on linguistic similarity measures for 263 languages . the results show that the effect of linguistic similarities on transfer performance depends on a range of factors . |
Adaptive Cross-lingual Text Classification through In-Context One-Shot Demonstrations (2024.naacl-long)
Copied to clipboard
| Challenge: | Zero-Shot Cross-lingual transfer (ZS-XLT) uses a model trained in a source language to make predictions in another language, often with a performance loss. |
| Approach: | They propose a new approach that uses In-Context Tuning to train a model to learn from context examples and adapt it to a target language by prepending a One-Shot context demonstration. |
| Outcome: | The proposed approach outperforms prompt-based models in Zero-Shot and Few-shot scenarios with target-language examples. |
Cross-Lingual Training for Automatic Question Generation (P19-1)
Copied to clipboard
| Challenge: | Automatic question generation is a challenging problem in natural language understanding . manual curating a dataset of comparable size for a new language is tedious and expensive. |
| Approach: | They propose to reuse available large QG dataset in a secondary language to learn a QG model for a primary language. |
| Outcome: | The proposed model outperforms baseline models in Hindi and Chinese. |
Multilingual and Cross-Lingual Graded Lexical Entailment (P19-1)
Copied to clipboard
| Challenge: | a novel method for capturing graded (and binary) LE is developed for cross-lingual generalisation of lexical entailment . lexicale enlargement is a key principle behind hierarchical structure found in semantic networks . |
| Approach: | They propose a method for cross-lingual generalisation of GR-LE relation using hyperlex and a bilingual dictionary. |
| Outcome: | The proposed method outperforms current state-of-the-art on binary cross-lingual LE detection by a wide margin. |
Good Meta-tasks Make A Better Cross-lingual Meta-transfer Learning for Low-resource Languages (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Model-agnostic meta-learning has garnered attention as a promising technique for enhancing few-shot cross-lingual transfer learning in low-resource scenarios. |
| Approach: | They propose a Meta-Task Collector-based Cross-lingual Meta-Transfer framework to adapt data selection strategies to construct cross-lingual meta-tasks to reduce language gaps. |
| Outcome: | The proposed framework significantly improves model performance in the target language with minimal annotation costs. |
Boosting Cross-Lingual Transfer via Self-Learning with Uncertainty Estimation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent pre-trained language models have achieved remarkable zero-shot performance . we propose a self-learning framework that utilizes unlabeled data of target languages . |
| Approach: | They propose a self-learning framework that utilizes unlabeled data of target languages to select silver labels for cross-lingual transfer tasks. |
| Outcome: | The proposed framework outperforms baseline models on two cross-lingual tasks by 10 F1 on average and 2.5 accuracy on natural language inference (NLI). |
Taxonomy of Problems in Lexical Semantics (2023.findings-acl)
Copied to clipboard
| Challenge: | Semantic tasks are rarely formally defined, and the exact relationship between them is unclear . a taxonomy of several problems in lexical semantics is proposed to clarify this . |
| Approach: | They propose a taxonomy that elucidates the connection between several problems in lexical semantics . they propose equivalence theory and algorithmic problem reductions to reduce problems to word sense disambiguation (WSD) |
| Outcome: | The proposed taxonomy proves that word sense disambiguation and word synonymy are theoretically equivalent. |
Cross-Lingual Generalization and Compression: From Language-Specific to Shared Neurons (2025.acl-long)
Copied to clipboard
| Challenge: | Existing evidence suggests that multilingual language models can transfer knowledge across languages without explicit cross-lingual supervision. |
| Approach: | They analyze the parameter spaces of three multilingual language models to examine their representations . they find that models evolve from language-specific representations to more specialized layer functions . |
| Outcome: | The proposed model can generate coherent English text, rather than spanish text, and it can generate generalized representations, the authors show. |
Cross-Lingual Multi-Hop Knowledge Editing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on knowledge editing in monolingual settings focused on a single language, but there are significant gaps in performance between the two settings. |
| Approach: | They propose a cross-lingual multi-hop knowledge editing paradigm for measuring and analyzing the performance of various SoTA knowledge editing techniques in a multilingual setup. |
| Outcome: | The proposed system improves on previous methods in a cross-lingual setting and in English. |
Cross-lingual Linking of Automatically Constructed Frames and FrameNet (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing semantic frame resources have been manually elaborated, but manual development is labor-intensive. |
| Approach: | They propose to link Japanese frames to English FrameNet by using cross-lingual word embeddings and a model that takes only the frame-evoking words into account. |
| Outcome: | The proposed model will facilitate the development of cross-lingual frame resources. |
Cross-lingual Emotion Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | Emotion detection is a useful tool for understanding human behavior, but constructing annotated datasets to train models can be expensive. |
| Approach: | They propose to use English as the source language with Arabic and Spanish as target languages to train models for emotion detection in a target language. |
| Outcome: | The proposed approaches surpass state-of-the-art models in Arabic and Spanish by 4% and 5% respectively. |
Improving Cross-lingual Transfer with Contrastive Negative Learning and Self-training (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent studies improve cross-lingual transfer learning by better aligning the internal representations within the multilingual model or exploring the information of the target language using self-training. |
| Approach: | They propose to use negative pairs to align the multilingual model and self-train the model to converge on the obtained clean pseudo-labels. |
| Outcome: | The proposed method improves upon the baseline models and can serve as a beneficial complement to the alignment-based methods. |
Can Cross-Lingual Transferability of Multilingual Transformers Be Activated Without End-Task Data? (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for cross-lingual transfer learning cannot activate cross-linguistic transferability when end-task data are unavailable. |
| Approach: | They propose a cross-lingual transfer method that disassembles multilingual Transformers into sub-modules and reassembles them to be the multilingual end-task model. |
| Outcome: | The proposed method activates the cross-lingual transferability of multilingual Transformers without accessing end-task data. |
One For All & All For One: Bypassing Hyperparameter Tuning with Model Averaging for Cross-Lingual Transfer (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot cross-lingual transfer are unreliable due to the lack of pretraining data. |
| Approach: | They propose to accumulatively average model snapshots from different runs into a single model. |
| Outcome: | The proposed protocol decouples performance maximization from hyperparameter tuning. |
X-SNS: Cross-Lingual Transfer Prediction through Sub-Network Similarity (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Cross-lingual transfer (XLT) is an emergent ability of multilingual language models that preserves their performance when evaluated in non-English languages. |
| Approach: | They propose to use sub-network similarity between two languages as a proxy for XLT prediction. |
| Outcome: | The proposed method shows proficiency in ranking candidates for zero-shot XLT, achieving an improvement of 4.6% on average in terms of NDCG@3. |
Data Contamination Can Cross Language Barriers (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect contamination of public benchmarks are too superficial to reflect deeper forms of contamination. |
| Approach: | They propose generalization-based approaches to unmask a cross-lingual form of contamination that inflates LLMs’ performance while evading current detection methods. |
| Outcome: | The proposed model outperforms existing detection methods while avoiding contamination of public benchmarks in the pre-training data. |
A Reinforcement Learning Framework for Cross-Lingual Stance Detection Using Chain-of-Thought Alignment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to cross-lingual stance detection can't effectively perform cross-linguistic transfer of complex reasoning processes. |
| Approach: | They propose a framework to facilitate cross-lingual transfer of complex reasoning processes in stance detection by using cross-linguistic Chain-of-Thought alignment to obtain high-quality CoTs generated from target language inputs. |
| Outcome: | The proposed framework outperforms competing models on four multilingual datasets. |
Understanding LLMs’ Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From (2025.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual context retrieval is a fundamental aspect of cross-lingual alignment, but the performance and mechanism of it for large language models (LLMs) remains unclear. |
| Approach: | They evaluate cross-lingual context retrieval of over 40 large language models . they use cross-linguistic machine reading comprehension as a representative scenario . |
| Outcome: | The results show that open LLMs show strong cross-lingual context retrieval ability . the results also show that their oracle performances improve after training . |